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FIELD OF THE INVENTION 

The invention relates to a video encoder and a method for predictively encoding 
images with reference to one reference image in a first prediction mode and to two reference 
images in a second prediction mode. 



BACKGROUND OF THE INVENTION 

Predictive video encoders as defined in the opening paragraph are generally 
known. For example, the MPEG video compression standard provides a first prediction mode 
in which imaiges are encoded with reference to a previous image of the sequence. Thus 

1 0 encoded images are referred to as P-pictures. The previous image may be an autonomously 
encoded I-picture or another P-picture. The MPEG standard also provides a second prediction 
mode in which images are encoded with reference to a previous image as well as a subsequent 
image of the sequence. Thus encoded images are referred to as B-pictures. They are more 
efficiently encoded than P-pictures. 

1 5 Motion compensation is usually applied to the respective reference images. To 

this end, the known video encoders include a motion estimation circuit which searches motion 
vectors representing motion between the input image and respective reference image(s). In the 
first (P) prediction mode, the motion estimation circuit is used for searching forward motion 
vectors representing motion between input im^e and previous image. In the second (B) 

20 prediction mode, the motion estimation circuit is used in a first interval of the frame encoding 
period for searching forward motion vectors representing motion between input image and 
previous image, and in a second interval of said frame encoding period for searching backward 
motion vectors representing motion between input image and subsequent image. 



25 OBJECT AND SUMMARY OF THE INVENTION 

It is an object of the invention to further improve the video encoder. 
To this end, the video encoder in accordance with the invention is characterized 
in that the motion estimation circuit is arranged to use the first interval of the frame encoding 
period in the first (P) prediction mode to search motion vectors representing motion between 
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an input image and said one reference image, and use the second interval of said frame 
encoding period to refine the motion vectors found in the first interval. 

The motion estimation circuit of the prior art video encoder executes the motion 
vector search twice within a fi:'ame encoding period in the second (B) prediction mode. One 
run of the search process is used to generate the forward motion vectors, another run is used to 
generate the backward motion vectors. In the first (P) prediction mode, forward motion vectors 
need be searched only. The invention exploits the insight that the motion estimator can be used 
a second time in the P-prediction mode so as to fiirther refine the search for forward motion 
vectors. With the invention is achieved that the motion vectors associated with P-pictures are 
more precise than the motion vectors associated with B-pictures. This is attractive because 
P-pictures are generally wider apart from each other than B-pictures. 

In an embodiment of the encoder, which the motion estimation circuit is 
arranged to search a motion vector from among a plurality of given candidate motion vectors, 
said candidate motion vectors in the second intervsd are formed by predetermined variations of 
the motion vector foxmd in the first interval. 

BRIEF DESCRIPTION OF THE FIGURES 

Fig. 1 shows a schematic diagram of a video encoder in accordance with the 

invention. 

Fig. 2 shows a diagram to illustrate a prior art operation of the video encoder. 
Figs. 3 and 4A-4C show diagrams to illustrate the operation of the video 
encoder in accordance with the invention. 

DESCRIPTION OF EMBODIMENTS 

The invention will now be described with reference to an MPEG video encoder. 
However, the invention is not restricted to encoders complying with the MPEG standard. 
Fig, 1 shows a schematic diagram of the encoder in accordance with the invention. The 
general layout is known per se in the art. The encoder comprises a subtracter 1, an orthogonal 
transform (e.g. DCT) circuit 2, a quantizer 3, a variable-length encoder 4, an inverse quantizer 
5, an inverse transform circuit 6, an adder 7, a memory imit 8, and a motion estimation and 
compensation circuit 9. 

The memory imit 8 includes memories 81a £ind 81b each having the capacity 
for storing a reference image. The memory unit further includes two switches 83 and 84. 
Switch 83 controls which one of the memories is read by the motion estimator. Switch 84 
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controls in which one of the memories the encoded image is stored. The switches are 
implemented as software controlled memory addressing operations in practical embodiments 
of the encoder. 

As is generally known in the art of video coding, I-pictures are autonomously 
5 encoded without reference to a previously encoded image. Subtracter 1 is inoperative. The 
I-picture is locally decoded and stored in one of the memories 81a or 81b, P-pictures and 
B-pictures are predictively encoded with reference to a prediction image. To this end, the 
subtracter 1 subtracts a motion-compensated prediction image Xp from the input image Xi, so 
that the difference is encoded and transmitted. Adder 7 adds the locally decoded P-picture to 
1 0 the prediction image so as to update the stored reference image. 

Fig. 2 shows a timing diagram to summarize a prior art operation of the 
encoder. The diagram shows the positions of switches 83 and 84 during consecutive frame 
periods for encoding an IBBPBBP.. sequence. The frames are identified by encoding type (I, 
B, P) and display order. II is the first frame, B2 is the second frame, B3 is the third fi-ame, P4 
15 is the fifth fii^me, etc. 

More particularly. Fig. 2 shows that the I-picture (II) is written in memory 81a 
(switch 84 in position a). The first P-picture (P4) is predictively encoded with reference to the 
stored I-picture (switch 83 in position a), and written into memory 81b (switch 84 in position 
b). The subsequent P-pictures (P7, PIO, ,.) are alternately read from and written in the 
20 memories 81a and 81b. 

B-pictures are encoded with reference to a previous and a subsequent I- or 
P-picture. Note that this requires the encoding order of images to be different from the display 
order. Circuitry therefor is known in the art and not shown in the Figure. The motion 

i 

estimation and compensation circuit 9 accesses both memories 81a and 81b to generate 
25 forward motion vectors (referring to the previous image) and backward motion vectors 

(referring to the subsequent image). To this end, the switch 83 SAvitches between position a and 
position b. Said switching is shown on a frame-by-frame basis for simplicity. In practice, the 
switching is done at the macroblock level. 

The motion estimation circuit executes a given motion vector search process. 
30 Said process requires reading of the respective memory a given nimiber of times, say N. As 
Fig. 2 clarifies, encoding of B-pictures requires 2N memory accesses per frame period. The 
same vector search process requires N memory accesses per frame in the P-encoding mode, 
whereas 2N accesses are available. This recognition is exploited by the invention. To this end, 
the motion vector search process is carried out in two passes for P-pictures. In the first pass, 
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the motion vectors are found with a 'standard* precision. In the second pass, the same process 
is carried out to further refine the accuracy of the motion vectors that were foxmd in the first 
pass. The two-pass operation is illustrated in Fig. 3, the refining pass being denoted by a* or b* 
as the case may be. Note again that the two-p£iss operation is carried out on a macroblock-by- 
5 macroblock basis in practice. 

Figs. 4A-4C show parts of an image to further illustrate the two-pass motion 
estimation process. Fig, 4A shows a cxirrent image 400 to be predictively (P) encoded. The 
image is divided into macroblocks. A current macroblock to be encoded includes an object 
401. Reference numerals 41, 42, 43 and 44 denote motion vectors already found during 

10 encoding of neighboring macroblocks. Figs. 4B and 4C show the previous I- or P-picture 402 
stored in one of the memories 81a or 81b as the case may be. In the previous reference image, 
the object (now denoted 403) is at a different position and has a slightly different shape. In this 
example, the motion estimator searches the best motion vector firom among a number of 
candidate motion vectors. Various strategies for selecting suitable candidate motion vectors 

15 are known in the art. It is here assumed that the motion vectors denoted 41, 42, 43 and 44 in 
Fig. 4 A are among the candidate motion vectors for the current macroblock Fig. 4B shows the 
result of the first motion vector search process pass. It appears that candidate motion vector 43 
provides the best match between the current macroblock of the input image and an equally 
sized block 404 of the reference image. 

20 In the second pass, the same search algorithm is applied with different 

candidate vectors. More in particular, the motion vector found in the first pass is one candidate 
motion vector. Other candidate vectors are fiorther refinements thereof. This is illustrated in 
Fig. 4C, where 43 is the motion vector foimd in the first pass and eight dots 45 represent end 
points of new candidate motion vectors. They differ from motion vector 43 by one (or one- 

25 half) pixel. The same search algorithm is now carried out with the new candidate vectors. It 
appears in this example that block 405 best resembles the current macroblock. Accordingly, 
motion vector 46 is the motion vector which is used for producing the motion-compensated 
prediction image Xp. The two-pass operation for P-pictures is particidarly attractive because it 
provides more accurate motion vectors for images that are wider apart than B-pictures. 

30 The invention can be summarized as follows, A conventional MPEG video 

encoder searches forward motion vectors with respect to a previous image and backward 
motion vectors with respect to a subsequent image in order to provide a motion-compensated 
prediction image for encoding B-pictures. This requires 2N accesses to the memory in which 
said images are stored. Searching the motion vectors for P-pictures requires N memory 
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accesses. The invention uses the spare capacity by running a two-pass motion vector search in 
the P-coding mode. In the second pass, the precision of the motion vectors foxmd in the first 
pass is further refined. This provides more accurate motion vectors for P-pictures. 
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CLAIMS: 



5 

1 . A video encoder for predictively encoding images with reference to one 
reference image in a first (P) prediction mode and to two reference images in a second (B) 
prediction mode, comprising a motion estimation circuit xising a first interval of a firame 
encoding period in the second (B) prediction mode to search motion vectors representing 

10 motion between an input image and one of said two reference images, and using a second 

interval of said frame encoding period to search motion vectors representing motion between 
said input image and the other one of said two reference images; characterized in that the 
motion estimation circuit is arranged to use the first interval of the frame encoding period in 
the first (P) prediction mode to search motion vectors representing motion between an input 

1 5 image and said one reference image, and use the second interval of said frame encoding period 
to refine the motion vectors found in the first interval. 

2. A video encoder as claimed in claim 1, in which the motion estimation circuit is 
arranged to search a motion vector from among a plurality of given candidate motion vectors, 

20 said candidate motion vectors in the second interval being formed by predetermined variations 
of the motion vector foxmd in the first interval. 

3. A video encoder as claimed in claim 1 or 2, wherein said reference image in the 
first prediction mode is a previous image of a sequence of images, one of the reference images 

25 in the second prediction mode is a previous image of said sequence, and the other one of the 
reference images in the second prediction mode is a subsequent image of said sequence. 

4. A method of predictively encoding images with reference to one reference 
image in a first (P) prediction mode and to two reference images in a second (B) prediction 

30 mode, comprising the steps of searching motion vectors representing motion between an input 
image and one of said two reference images in a first interval of a fi*ame encoding period in 
the second (B) prediction mode, and searching motion vectors representing motion between 
said input image and the other one of said two reference images in a second interval of said 
frame encoding period; characterized by the steps of searching motion vectors representing 
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motion between an input image and said one reference image in the first interval of the frame 
encoding period in the first (P) prediction mode, and refining the motion vectors foimd in the 
first interval in the second interval of said frame encoding period. 



5 5, A method as claimed in claim 4, in which the step of searching a motion vector 

comprises selecting a motion vector from among a plurality of given candidate motion vectors, 
said candidate motion vectors in the second interval being formed by predetermined variations 
of the motion vector foxmd in the first interval. 



10 6. A method as claimed in claim 4 or 5, wherein said reference image in the first 

prediction mode is a previous image of a sequence of images, one of the reference images in 
the second prediction mode is a previous image of said sequence, and the other one of the 
reference images in the second prediction mode is a subsequent image of said sequence. 
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5 

A conventional MPEG video encoder searches forward motion vectors vnth 
respect to a previous image and backward motion vectors with respect to a subsequent image 
in order to provide a motion-compensated prediction image for encoding B-pictures. This 
reqvdres 2N accesses to the memory in which said images are stored. Searching the motion 
10 vectors for P-pictures requires N memory accesses. The invention uses the spare capacity by 
running a two-pass motion vector search in the P-coding mode. In the second pass, the 
precision of the motion vectors found in the first pass is further refined. This provides more 
accurate motion vectors for P-pictures, 

15 Fig. 1. 
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